Remote dynamic configuration of a web server to facilitate capacity on demand

ABSTRACT

The present invention relates to a method and system for the remote and dynamic configuration of a webserver to facilitate capacity on demand. The present invention provides a solution to the problem of the management and administration of one or more servers within an environment such as a server farm with regards to the rigid infrastructure and architecture of the servers due to the definition of roles the servers play in relation to the data the servers are publishing. A solution is provided in the form of autonomic modular computing components to allow a server to process a URL and the server to associate it with a remote data source without the need for a restart or manual intervention of the server and to allow the introduction of new services and or hardware resources by collating and analysing performance data collected from the server to determine if the server is over or under utilised.

FIELD OF THE INVENTION

The invention relates to the field of network services and in particularto the remote and dynamic configuration of a server to facilitatecapacity on demand.

BACKGROUND OF THE INVENTION

Many companies offer hosting services to provide customers with asecure, robust and flexible infrastructure in which to host a variety ofapplications for example web applications such as on-line banking,on-line shopping, information services and hosting service such as ‘payfor used capacity’ which, enable a customer to only pay for theprocessing power that they use and allows the customer to deploy themost up-to-date equipment in a cost effective manner. Hosting servicesprovide many businesses with an alternative solution to building andrunning their technology infrastructure in-house by tapping intocomputer systems in other company's data centers to provide themanagement of software applications and hardware resources such asservers. The management and administration of these servers and servicesprovide a tremendous challenge to many hosting companies, as a keyproblem with the management and administration of the servers within anenvironment such as a server farm is the rigid infrastructure andarchitecture of the servers due to the definition of roles the serversplay in relation to the data the servers are publishing.

The rigid allocation of a resource to for example, a web server whichsupports a particular customer's product can result in an under utilizedyet expensive web server resource not being used to it's fullcapability, while other web servers supporting other products arestretched to the point of breaking. Further today's current state of theart web server software is complex and flexible and can be configuredand extended through Application Protocol Interfaces (API) to facilitatepowerful processing beyond the standard serving of basic HypertextMarkup Language (HTML) pages. However current deployment practices andtechnologies require that the web servers are configured manually usingeither configuration files or binary registries and thus remain staticduring their operation, handling requests for a specific website orwebsites. When an additional or a different resource for a particularHTML page or a different web site URL needs publishing or a need arisesfor a new server to be added to, or removed from a server pool, theservers must be restarted for any change to take effect. This requires atremendous administration effort on the part of the server farmadministrator because it takes a considerable amount of time to manuallyconfigure a server and it is not always convenient to shut down a serverand restart the server to allow for any changes to take effect as thiscan cause loss of service for a period of time.

US patent application publication US 2002/0002602 (2602) describes asystem for serving web pages to a client in response to a client requestspecifying a resource that aims to serve a web page in a coordinatedfashion from multiple cooperating web servers and maintaining a reliableconnection so that the server and clients remain synchronized andinformation is not lost. In order for the above to take place such thata web server processes a URL and associates the URL with a data source,the web server will require manual intervention and the web server willhave to be shut down and restarted for the changes to take effect.

DISCLOSURE OF THE INVENTION

In accordance with the present invention there is now provided a methodfor the remote and dynamic configuration of a server to facilitatecapacity on demand comprising the steps of: (a) a client devicerequesting a resource from a first server in a communications network;(b)the first server receiving the request for the resource from theclient device; (c) the first server routing the client request for theresource to a dynamic content module, the dynamic content moduleidentifying an available third server from which the requested resourcecan be served and routing the requested resource to the client device;(d) collating performance data from the first and third server and thefirst server reporting the performance data to a second server; (e) asecond server analysing the performance data collated in step (d) todetermine performance capabilities of the first and the third server andidentifying if the first or the third server has reached a predeterminedthreshold; and (f) the second server adjusting the allocation of thefirst server or the third server in response to step (e) and issuing aconfiguration update instruction for the first server or the thirdserver to a dynamic configuration module of the first server anddetermining if a resource update is successful.

The present invention advantageously allows for the optimisation of aservers performance such that a server can be allocated or deallocatedfrom a server pool depending on the server's performance for example ifa server is stretched to capacity or equally if a server is underutilised. Further using modular autonomic computing components thesystem is able to configure and reconfigure itself under varying andunpredictable conditions. Further the method allows for making a changeto a hardware resource or a data resource without the need for theserver to be restarted as with other prior art systems.

Preferably the invention provides for the dynamic content modulerequesting a connection configuration file for the third server from thedynamic configuration module. This allows for the first server todetermine which data source server has the resource requested by theclient.

Preferably the invention provides for the adjusting of the allocation ordeallocation of one or both the first server and the third server from afree server pool or to a free server resource pool. The presentinvention can therefore monitor a servers performance and either add anadditional resource or de-allocate a resource depending on the work loadof the server.

Preferably the invention provides for the requested resource beingdecoupled from the first server allowing the introduction of a newservice or the removal of a redundant service.

Viewed from another aspect the present invention provides a system forthe remote and dynamic configuration of a server to facilitate capacityon demand, the system comprising a client device for requesting andreceiving a resource in a communications network, the system furthercomprising:

a first server, the first server comprising a dynamic content module, adynamic configuration module and a reporting module; means for routingthe client request for the resource to the dynamic content module; meansfor the dynamic content module identifying a third server from which therequested resource can be served and means for retrieving a connectionconfiguration file associated with the third server stored in thedynamic configuration module; means for the reporting module collatingperformance data from the first server and the third server and meansfor routing the performance data to a second server;

a second server comprising an analyser module, a resource allocationmodule and a resource update module, the second server sending theperformance data to the analyser module; means for the analyser moduledetermining the performance capabilities of the first server and thethird server and means for identifying if the first server and thirdserver has reached a predetermined threshold; means for the resourceallocation module adjusting the allocation of one or both the firstserver and the third server in response to the identifying means; meansfor the resource update module issuing an configuration updateinstruction for one or both the first server and the third server to thedynamic configuration module of the first server and means fordetermining if a resource update is successful; and a third servercomprising one or more resources and means for the third server to servea requested resource to the first server.

A another advantage is that an HTTP URL can be processed by a server andassociated with a remote data source without the need for the server tobe restarted or requiring manual intervention. A further advantage ofthe present invention is the secure and centralised administration forthe dynamic plug and play of data sources and hardware resources suchthat a web site can be served ‘on the fly’. Another advantage of thepresent invention is for the provision of multiple protocol support forother mechanisms such as FTP, XML, SOAP and file sharing.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described by way of example only, withreference to the accompanying drawings, in which:

FIG. 1 illustrates a server farm in which the present invention may beimplemented in accordance with a preferred embodiment of the presentinvention;

FIG. 2 illustrates a block diagram detailing an overview of thecomponents of the system in accordance with a preferred embodiment ofthe present invention;

FIG. 3 illustrates a flowchart detailing the function of the dynamiccontent module of the web server of FIG. 2 in accordance with apreferred embodiment of the present invention;

FIG. 4 illustrates a flowchart detailing the function of the reportingmodule of the web server of FIG. 2 in accordance with a preferredembodiment of the present invention;

FIG. 5 illustrates a flowchart detailing the function of the analysermodule of the management server of FIG. 2 in accordance with a preferredembodiment of the present invention;

FIG. 6 illustrates a flowchart detailing the function of the resourceallocation module of the management server of FIG. 2 in accordance witha preferred embodiment of the present invention; and

FIG. 7 illustrates a flowchart detailing the function of resource updatemodule of the management server of FIG. 2 in accordance with a preferredembodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE INVENTION

FIG. 1 illustrates a server farm in which the present invention may beimplemented. The server farm includes a plurality of client devices 100and 105 which are connected to a network such as the Internet 110. Theclient devices 100 and 105 perform transactions by sending and receivingrequests for a resource over the network 110. Routers 115 and 120provide a connection between the network 110 and the local area network(LAN) 125 which could be a high speed Ethernet network or any othersuitable transmission medium or topology. The routers 115 and 120receive an incoming request and forward the request over a network 135to a router/firewall 130 which filters the request to a server 145 to155. The present invention embodies a management server 140 for thecentral control point for servers 145 to 155. The servers 140 to 155 mayimplement one or more server technologies including, for example UNIXwhich is a registered trademark of The Open Group in the United Statesand other countries, Novell which is a registered trademark of NovellInc in the United States and other countries, or Windows NT which is aregistered trademark of Microsoft Corporation in the United States andother countries or both, and/or a peer to peer networking arrangement.Although the present invention has been described with reference to aserver farm, the present invention could equally be applicable topervasive computing environments as well as other server networkingarrangements.

Application data is stored in a data store in one or more data sourceservers 175 and 180 which are separate from the servers 140 to 155. Arequest is sent from any one of the servers 145 to 155 across a network160. The request is received by a router/firewall 165 and routed over afurther network 170 to the appropriate data source server 175 and 180storing the resource that was requested by one or more client devices100 and 105.

Referring to FIG. 2, a schematic view of the system is shown. The systemcomprises modular autonomic computing components to provide selfdiagnostic capabilities to detect performance issues with servers 145 to155. Further by using modular components the system is able to providethe ‘plug and play’ of data sources for the easy introduction of newservices.

The system comprises a server 145 with a reporting module 205 forcollating performance and request data from each server 145 to 155 in anallocated server pool; a dynamic content module 215 which routes therequest for a resource through the most appropriate interface to accessa requested resource either using the native data access capability ofthe server 145 to 155 or by establishing and managing the connection toa remote server itself; a dynamic configuration module 216 for theupdate, processing and storing of a configuration file for each serverin the allocated server pool and a client device 100 for requesting aresource from the server. The configuration file is used to hold theconfiguration settings for a particular server 145 to 155 for example aregistration key, port settings, an installation path, authenticationsettings, user settings, SMPT settings, connection settings, web browsersettings and log in settings.

The system further comprises a management server 140 for providing acentral control point for all servers in an available server pool; ananalyser module 230 for analysing the performance and request data fromthe reporting module 205; a resource allocation module 235 to determinewhether there is a free resource in the free server resource pool to beadded to the pool of allocated servers or if necessary to deallocate aserver and place it back in the free server resource pool; a resourceupdate module 240 for determining whether a resource has beensuccessfully allocated or deallocated to or from the available resourcepool and a management client 245 for the entry of manual configurationtasks into the system. The management client 245 can receiveadministration requests from a server administrator. The managementclient functions as a graphical user interface and provides greaterflexibility in terms of providing connectivity to the management serverfor example providing connectivity remotely in another site via theInternet using a standard HTTP connection between the servers 145 to 155and the management server 140. To enable only authorised administratorsto make configuration changes, an interface may be provided to asecurity directory for handling authentication and entitlement requests(not shown in the figures).

Data streams 220 and 225 are sent to and from the server 145 to 155 andthe management server 140 using an Extensible Markup Language (XML) asthe transport mechanism 220 and 225 over an HTTP session. XML is theuniversal format for structured documents and data on the Web. XML usestags similar to HTML, and allows data to be structured in a manner thatis understood by many people. Further information on XML can be found atthe World Wide Web Consortium.

The client device 100 sends a request for a specific resource to theserver 145. One such request that may be recieved by a server 145 to 150is a HTTP URL for example http://www.ibm.com requesting an HTML page. Toallow the server to translate the HTTP URL, the server will use a DomainName Service (DNS) which translates Internet domain names to networkaddresses such as 129.42.19.99. The resource could be a standard HTMLpage or an .exe file. The server 145 receives the request for theresource from the client device 100 and 105 and the request is sent tothe dynamic content module 215.

Although FIG. 2 illustrates a management server 140 and a server 145, itshould be appreciated that the system may comprise a plurality ofservers 145 to 155 as shown in FIG. 1, each server containing themodules as shown in FIG. 2.

The operation of the dynamic content module 215 will now be explainedwith reference to FIG. 3. It should be noted that the dotted line inFIG. 3 represents data flow and the solid single line represents controlflow. The dynamic content module 215 receives the request for a resourcefrom the client device at step 300 and compares the content stored onthe server 145 with the requested resource at step 305. For example if aclient requested a resource from www.ibm.com/news/today.html, thedynamic content module compares the today.html page stored on the serverwith the today.html page stored on a source server to determine if thetoday.html page has been updated therefore, sending the client devicethe most up to date today.html page. At step 310 the server determineswhether the content has been cached and if the cached content is currentand up to date at step 360. If the cache is current control flows tostep 355 and the requested resource is retrieved from the content cachestore 350, equally if the content is not cached, the server whichcontains the requested resource is identified at step 315. Further atstep 355 information concerning the server's 145 activities is sent tothe reporting module 205. The information can include data such as thenumber of requests for each resource such as a product, the time ittakes the server to respond or the type of service requested such asHTTP or File Transfer Protocol (FTP).

To enable the server 145 to identify the data source server 175 theserver retrieves the connection configuration file for the source serverat step 325 from a connection configuration store 320. Preprocessing isperformed on the content request at step 330. At step 335 a request isissued to the data source server 175 and at step 340 the server 145receives the content as requested by the client device 300 from thesource server. The retrieved content is either stored in the contentcache data store 350 or the existing content is refreshed with theretrieved content at step 345 and stored in the content cache data store350.

Referring back to step 355, information regarding the resource requestedfrom the client device 100 at step 300 along with performanceinformation is sent to the reporting module 210. With reference to FIG.4 the reporting module 205 receives the content information from thedynamic content module 215 and records the information at step 400.These details are stored in a server statistics data store at step 405and the server statistics (response times, number of pages requested,how many times, what pages, which server, server config details andperformance data) associated with each server 145 to 155 are sent to themanagement server 140 at step 410 via an XML data streams 220.

Referring to FIG. 5, the analyser module 230 of the management server140 receives the server statistics in the XML data streams 220 and atstep 500 the analyser module 230 analyses the server statistics todetermine whether the server 145 to 155 has reached a predeterminedthreshold at step 505, if the server's performance has not reached thepredetermined threshold control flows back to step 500, else controlflows to step 510 and the analyser module 230 analyses the server'scurrent configuration settings and the server's configuration settingsare sent to the resource allocation module 235.

Referring to FIG. 6 at step 600, the resource allocation module 235determines whether a new server should be added to a current pool ofallocated severs 145 to 155 if the performance threshold has beenexceeded or if a server should be removed from the current pool ofallocated servers and returned to the free resource pool. If theanalyser module 230 determines that an additional server is to be added,control flows to step 605 and the resource allocation module identifieswhether there exists a spare resource at step 610. A spare resourcecould be for example an additional server or equally some other hardwareresource. If a spare resource can not be located control flows to step630 and at step 635 an exception error is created and displayed in themanagement console 245 to alert a technician that the resource updatehas failed at step 635. Referring back to step 610 if a spare resourcehas been located control flows to step 615 and a new server is selectedand information regarding the server's configuration settings file isrequested from the configuration database 615 of the particular server140 to 155. At step 620 once the configuration setting file has beenreceived, the configuration setting file is updated with the currentinstruction for the selected server and sent to the resource updatemodule 240 and the dynamic configuration module 216 via XML data streams225.

Referring to FIG. 7, at step 700 the configuration update information isissued to the dynamic configuration module 216 of the server 145 via XMLdata streams 225. The resource update module 240 waits foracknowledgment from the server 145 at step 710. If a configurationupdate is successful at step 715 control flows to step 720 and theconfiguration database 730 is updated. Referring back to step 720 if theacknowledgment from the server 145 is successful control also passes tostep 735 and a configuration update request for a load balancinginfrastructure is requested. Equally if at step 720 the update is notsuccessful control flows to step 745 and an exception handling alert isgenerated and the resource update is deemed to have failed at step 750.

1-21. (canceled)
 22. A method for allocating servers to a cluster ofservers, said method comprising the steps of: sending performance dataof a first server to a second server, said first server being part ofsaid cluster of servers; based on the performance data, said secondserver determining if said first server has reached a predeterminedupper level of utilization; and if said first server has reached saidpredetermined upper level of utilization, said second serverautomatically sending a reconfiguration request to a server responsiblefor allocating servers to said cluster to allocate another server tosaid cluster, and in response, said responsible server automaticallyidentifying another, available server and connection information forsaid other server and allocating said other server to said cluster. 23.A method as set forth in claim 22 wherein the step of automaticallyallocating said other server to said cluster comprises the steps ofupdating a configuration file of said responsible server to list saidother server as part of said cluster.
 24. A method as set forth in claim22 further comprising the steps of: based on the performance data, saidsecond server determining if said first server is under utilized; and ifsaid first server is under utilized, said second server automaticallysending a reconfiguration request to said server responsible forallocating servers to said cluster to de-allocate said first server fromsaid cluster, and in response, said responsible server automaticallyde-allocating said first server from said cluster.
 25. A method as setforth in claim 24 wherein the step of automatically de-allocating saidfirst server from said cluster comprises the step of updating aconfiguration file of said responsible server to remove said firstserver from said cluster.
 26. A system for allocating servers to acluster of servers, said system comprising: means for sendingperformance data of a first server to a second server, said first serverbeing part of said cluster of servers; means, based on the performancedata, within said second server for determining if said first server hasreached a predetermined upper level of utilization, and if said firstserver has reached said predetermined upper level of utilization,automatically sending a reconfiguration request to a server responsiblefor allocating servers to said cluster to allocate another server tosaid cluster; and means, responsive to said reconfiguration request,within said responsible server for automatically identifying another,available server and connection information for said other server andallocating said other server to said cluster.
 27. A system as set forthin claim 26 further comprising: means, based on the performance data,within said second server for determining if said first server is underutilized, and if said first server is under utilized, automaticallysending a reconfiguration request to said server responsible forallocating servers to said cluster to de-allocate said first server fromsaid cluster; and means, responsive to the de-allocation reconfigurationrequest, within said responsible server for automatically de-allocatingsaid first server from said cluster.
 28. A method for managing servers,said method comprising the steps of: a first server determiningperformance data for said first server and performance data for a secondserver, and reporting to a third server said performance data for saidfirst server and said performance data for said second servers, saidfirst and second servers being in a cluster of servers; based on thereported performance data, said third server determining if said firstserver or said second server has reached a predetermined upper level ofutilization; and if said first server or said third server has reachedsaid predetermined upper level of utilization, said third server sendinga reconfiguration request to said first server to reduce subsequentutilization of the server which has reached said predetermined upperlevel of utilization, and said first server automaticallyreconfigurating itself to reduce subsequent utilization of the serverwhich has reached said predetermined upper level of utilization.
 29. Amethod as set forth in claim 28 further comprising the earlier steps of:said first server receiving a request from a client device anddetermining whether said first server should handle said request, and ifso, said first server handling said request, and if not, said firstserver identifying said second server as available to handle saidrequest and forwarding the request to said second server for handling.30. A method as set forth in claim 28 wherein the step of said firstserver automatically reconfigurating itself comprises the step of saidfirst server updating a configuration file which lists one or moreservers which are available to handle specified types of clientrequests.
 31. A method as set forth in claim 28 wherein the step of saidfirst server automatically reconfigurating itself comprises the step ofautomatically identifying a fourth, available server and connectioninformation for said fourth server and allocating said fourth server tosaid cluster.
 32. A method as set forth in claim 28 further comprisingthe steps of: based on said performance data, said third serverdetermining if said first server or said second server is underutilized; and if said first server or said third server is underutilized, said third server sending a reconfiguration request to saidfirst server to de-allocate the server which is under utilized, and saidfirst server automatically reconfiguring itself to de-allocate from saidcluster said server which is under utilized.
 33. A method as set forthin claim 32 wherein the automatic de-allocating reconfiguring stepcomprises the step of adding to a pool of available servers, said serverwhich is under utilized, such that if a server in said clustersubsequently reaches said predetermined upper level of utilization, thede-allocated server can be re-allocated to said cluster.
 34. A method asset forth in claim 28 wherein: said first server reports to said thirdserver said performance data for said first server and said performancedata for said second server using XML data streams; and said thirdserver sends said reconfiguration request to said first server using anXML data stream.
 35. A method as set forth in claim 29 wherein the stepof said first server determining whether said first server should handlesaid request comprises the step of said first server determining if saidrequest is for data which is currently cached at said first server, andif so, the step of said first server handling said request comprises thestep of said first server supplying said data from the cache at saidfirst server to said client device.